{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h2c6PyqQaNiA"
      },
      "source": [
        "# Using the Learning Interpretability Tool in Notebooks\n",
        "\n",
        "This notebook shows use of the [Learning Interpretability Tool](https://pair-code.github.io/lit) on a binary classifier for labelling statement sentiment (0 for negative, 1 for positive).\n",
        "\n",
        "The LitWidget object constructor takes a dict mapping model names to model objects, and a dict mapping dataset names to dataset objects. Those will be the datasets and models displayed in LIT. Running the constructor will cause the LIT server to be started in the background, loading the models and datasets and enabling the UI to be served.\n",
        "\n",
        "Render the LIT UI in an output cell by calling the `render` method on the LitWidget object. The LIT UI can be rendered multiple times in separate cells if desired. The widget also contains a `stop` method to shut down the LIT server.\n",
        "\n",
        "Copyright 2020 Google LLC.\n",
        "SPDX-License-Identifier: Apache-2.0"
      ]
    },
    {
      "metadata": {
        "id": "k5QzhJlsq9_U"
      },
      "cell_type": "code",
      "source": [
        "# The pip installation will install all necessary prerequisite packages for use of the core LIT package.\n",
        "!pip install lit-nlp"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tthVhLoIq9_U"
      },
      "outputs": [],
      "source": [
        "from lit_nlp import notebook\n",
        "from lit_nlp.examples.glue import data\n",
        "from lit_nlp.examples.glue import models\n",
        "\n",
        "# Hide INFO and lower logs. Comment this out for debugging.\n",
        "from absl import logging\n",
        "logging.set_verbosity(logging.WARNING)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "30l9ZyTjxJjf"
      },
      "outputs": [],
      "source": [
        "# Fetch the trained model weights\n",
        "!wget https://storage.googleapis.com/what-if-tool-resources/lit-models/sst2_tiny.tar.gz\n",
        "!tar -xvf sst2_tiny.tar.gz"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AWhbAZg57RpB"
      },
      "outputs": [],
      "source": [
        "# Create the LIT widget with the model and dataset to analyze.\n",
        "datasets = {'sst_dev': data.SST2Data('validation')}\n",
        "models = {'sst_tiny': models.SST2Model('./')}\n",
        "\n",
        "widget = notebook.LitWidget(models, datasets, port=8890)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9GSfs1waBdLd"
      },
      "outputs": [],
      "source": [
        "# Render the widget\n",
        "widget.render(height=600)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dwYz8o1dq9_U"
      },
      "source": [
        "If you've found interesting examples using the LIT UI, you can access these in Python using `widget.ui_state`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HRtO27puq9_U"
      },
      "outputs": [],
      "source": [
        "widget.ui_state.primary  # the main selected datapoint"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "J-XT048tq9_U"
      },
      "outputs": [],
      "source": [
        "widget.ui_state.selection  # the full selected set, if you have multiple points selected"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6gxrwk-Bq9_U"
      },
      "outputs": [],
      "source": [
        "widget.ui_state.pinned  # the pinned datapoint, if you use the 📌 icon or comparison mode"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RndbhKC9q9_U"
      },
      "source": [
        "Note that these include some metadata; the bare example is in the `['data']` field for each record:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jbEVvUK6q9_U"
      },
      "outputs": [],
      "source": [
        "widget.ui_state.primary['data']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Pt3djyXPq9_U"
      },
      "outputs": [],
      "source": [
        "[ex['data'] for ex in widget.ui_state.selection]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "u03ZgGlAq9_U"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "LIT in Notebooks",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.15"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 1
}
